Multi-Fidelity Covariance Estimation in the Log-Euclidean Geometry

We introduce a multi-fidelity estimator of covariance matrices that employs the log-Euclidean geometry of the symmetric positive-definite manifold. The estimator fuses samples from a hierarchy of data sources of differing fidelities and costs for variance reduction while guaranteeing definiteness, in contrast with previous approaches. The new estimator makes covariance estimation tractable in applications where simulation or data collection is expensive; to that end, we develop an optimal sample allocation scheme that minimizes the mean-squared error of the estimator given a fixed budget. Guaranteed definiteness is crucial to metric learning, data assimilation, and other downstream tasks. Evaluations of our approach using data from physical applications (heat conduction, fluid dynamics) demonstrate more accurate metric learning and speedups of more than one order of magnitude compared to benchmarks.Comment: To appear at the International Conference on Machine Learning (ICML) 202

Multifidelity Covariance Estimation via Regression on the Manifold of Symmetric Positive Definite Matrices

We introduce a multifidelity estimator of covariance matrices formulated as the solution to a regression problem on the manifold of symmetric positive definite matrices. The estimator is positive definite by construction, and the Mahalanobis distance minimized to obtain it possesses properties which enable practical computation. We show that our manifold regression multifidelity (MRMF) covariance estimator is a maximum likelihood estimator under a certain error model on manifold tangent space. More broadly, we show that our Riemannian regression framework encompasses existing multifidelity covariance estimators constructed from control variates. We demonstrate via numerical examples that our estimator can provide significant decreases, up to one order of magnitude, in squared estimation error relative to both single-fidelity and other multifidelity covariance estimators. Furthermore, preservation of positive definiteness ensures that our estimator is compatible with downstream tasks, such as data assimilation and metric learning, in which this property is essential.Comment: 30 pages + 15-page supplemen

Multifidelity Covariance Estimation Three Ways

In this thesis we develop a suite of three methods for multifidelity covariance estimation. We begin with a straightforward extension of scalar multifidelity Monte Carlo to matrices, obtaining what we refer to as the Euclidean or linear control variate mutifidelity covariance estimator. The mean squared error of this estimator is available in closed form, which enables analytic optimization of sample allocations and weights to minimize expected squared Frobenius error subject to computational budget constraints. Despite its nice analytical properties and familiar closed-form construction, however, the Euclidean estimator can be subject to loss of positive-definiteness. Given this liability, we subsequently develop two multifidelity covariance estimators which preserve positive definiteness by construction, utilizing, to varying degrees, the geometry of the manifold of symmetric positive definite (SPD) matrices. Our first positive-definiteness-preserving estimator, referred to as the tangent space or log-linear control variate estimator, constructs a multifidelity covariance estimate by appplying linear control variates to sample covariance matrix logarithms, which are symmetric matrices residing in tangent spaces to the SPD manifold. Though the tangent space estimator preserves positive-definiteness and is straightforward to construct, obtaining its expected squared error, and thus choosing optimal sample allocations and control variate weights, are not tractable. When first-order approximations of the matrix logarithms involved are made, however, the optimal sample allocations and control variate weights for the tangent space estimator are the same as those of the Euclidean estimator, and in practice the tangent space estimator has been shown to yield variance reduction in example problems. In a departure from the control variate formulations of the Euclidean and tangent-space estimators, our third multifidelity covariance estimator is defined as the solution to a regression problem on tangent spaces to product manifolds of SPD matrices. Given a set of high- and low-fidelity sample covariance matrices, which we view as a sample of a product-manifold-valued random variable, we estimate the underlying true covariance matrices by minimizing an intrinsic notion of squared Mahalanobis distance between the data and a model for its variation about its mean. The resulting estimates are guaranteeably positive definite and the Mahalanobis distance which they minimize has desirable properties, including tangent-space agnosticism and affine-invariance. Mahalanobis distance minimization can be carried out using unconstrained gradient-descent methods when a reparametrization in terms of SPD matrix square roots is employed, and we introduce a new Julia package, CovarianceRegression.jl, providing a convenient API for solving these multifidelity covariance regression problems. Using its machinery, we demonstrate that our estimator can provide significant reductions in MSE over single-fidelity covariance estimators in forward uncertainty quantification problems.S.M